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SUMMARY 

Agile  aircraft  introduce  new  requirements  and 
performance  standards  for  the  pilot-vehicle  interface. 
This  lecture  will  address  these  ergonomic  issues  as 
they  pertain  to  agile  aircraft.  Specifically,  controls 
and  displays  will  be  discussed,  followed  by  design 
issues  relevant  to  intelligent  interfaces.  The  concepts 
and  technologies  proposed  as  candidate  solutions  for 
creating  pilot-vehicle  synergy  are,  for  the  most  part, 
untested  at  present.  It  is  hoped  that  this  lecture  will 
provide  tire  impetus  for  the  research  required  to 
realize  a pilot-vehicle  interface  that  will  enhance  the 
operation  of  agile  aircraft. 

6.1.  OVERVIEW 

Agile  aircraft  have  the  potential  to  provide  enhanced 
speed,  range,  flexibility,  and  lethality.  In  order  to 
exploit  these  benefits,  the  warfighter  must  be  able  to 
assess  situations,  decide  tactics  to  be  employed,  and 
execute  responses  under  rapid,  highly  uncertain  and 
temporally  demanding  combat  conditions. 
Unfortunately,  improvements  to  date  have  all  tended 
to  complicate  cockpit  design.  This  increased 
complexity  may  overload  pilots’  perceptual  and 
cognitive  processing  capabilities,  increase  workload, 
and  ultimately  degrade  mission  effectiveness. 

For  pilots  to  realize  the  benefits  afforded  by  agile 
aircraft,  crew  station  designs  must  facilitate  the 
potential  synergy  between  situational  awareness,  the 
maneuverability  envelope,  and  systems  [1].  For 
instance,  enhanced  maneuverability  will  not  increase 
survival  rates  if  pilots  do  not  realize  that  a change  in 
flight  path  is  recommended.  Moreover,  if  pilots  on  a 
flight  path  ending  with  ground  impact  have  real-time 
updates  of  their  situation,  they  can  choose  when  to 
alter  their  flight  path  - they  can  choose  whether  to 
change  the  path  immediately,  or  wait  until  the  final 
moment  the  envelop  allows  escape.  Likewise,  if 
pilots  are  cognizant  of  a threat,  but  weapon  selection 
is  time  consuming,  they  may  not  be  able  to  exploit 
the  advantages  of  increased  maneuverability.  On  the 
other  hand,  simple  and  direct  means  of  changing 
weapon  settings  may  achieve  a tactical  advantage 
without  arduous  maneuvering. 

These  examples  show  that  it  is  the  communication 
between  tire  crew  station  and  the  pilot  that  is  the 
limiting  factor  in  the  ability  of  pilots  to  exploit  the 
advantages  afforded  by  agile  systems.  Although 
there  are  some  specific  design  issues  presented  by 


new  capabilities,  it  is  the  multitude  of  systems  that 
constitute  agile  aircraft  that  make  the  pilots’ 
information  management  task  the  primary  challenge 
and  key  determinant  of  successful  deployment. 
Crew  station  design  with  the  goal  of  pilot-cockpit 
synergy  has  the  potential  to  provide  the  flexibility  to 
maximum  mission  effectiveness. 

6.2.  AGILE  AIRCRAFT  IMPLICATIONS 

The  pilot-vehicle  interface  used  in  agile  aircraft 
determines  how  fast  and  accurately  the  pilot  can 
assimilate  the  required  information  and  execute 
control  procedures.  Although  the  pilots  interviewed 
indicated  that  special  devices  are  not  required  to 
exploit  tlie  advantages  of  agile  airframes,  they  did 
raise  several  interface  design  issues  for  agile  aircraft 
systems.  These  are  discussed  below. 

6.2.1.  HIGH  ANGLE-OF-ATTACK  (AOA) 
Conventional  attitude  displays  can  not 
simultaneously  present  both  the  nose  position  and 
vertical  velocity  during  high  AOA  maneuvering. 
Moreover,  when  a pilot  is  recovering  from  a high 
AOA  maneuver  (e.g.,  over  45  degrees),  there  is  an 
initial  feeling  diat  die  aircraft  is  not  reducing  its 
AOA  in  response  to  nose  down  pitch  commands. 
This  makes  it  even  more  difficult  for  die  pilot  to 
maintain  awareness  of  the  flight  padi  vector.  Pilots 
need  a display  format  that  provides  a rapidly 
interpreted  indication  of  the  flight  path  response  for 
agile  aircraft. 

6.2.2.  NEW  COMBAT  MANEUVERS 

Agile  airframes  have  enabled  a new  range  of  combat 
maneuvers  (e.g.,  Herbst  Maneuver),  especially  since 
pilots  no  longer  have  to  point  die  aircraft’s  nose  in 
die  direction  of  the  target.  The  ability  to  rapidly 
change  flight  path  has  also  allowed  an  advantage 
during  flat  scissors  maneuvers.  These  two 
exemplary  maneuvers  present  corresponding  display 
challenges.  First,  the  pilot  must  receive  a clear 
indication  of  the  approach  of  a “terminal  exit  time”  - 
diat  point  in  flight  when  the  pilot  must  leave  die 
post-stall  domain  to  avoid  reaching  a hazardous 
aldtude.  Second,  additional  indication  of  yaw,  in 
addition  to  flight  path,  is  needed  for  die  pilot  to 
maintain  spatial  orientation.  A visual  reference  diat 
provides  an  accurate  orientation  cue  is  especially 
needed  during  automatic  guns  aiming  or  missile 
avoidance,  to  help  avoid  disorientation  and  sickness 
from  the  abrupt  maneuver  changes.  Also,  means  of 
maintaining  sight  and  situational  awareness  of  the 
target  during  high  angle-of-attack  flight  is  required. 


Paper  presented  at  the  RTO  HFM  Lecture  Series  on  “Human  Consequences  of  Agile  Aircraft”, 
held  in  Neubiberg,  Germany,  20-21  March  2000;  Preston,  UK,  23-24  March  2000; 
WPAFB  Ohio,  USA,  19-20  October  2000,  and  published  in  RTO-EN-12. 
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6.2.3.  HIGH  SPEED/EXPANDED  ENVELOPE 

The  anticipated  speed  that  can  be  achieved  by  agile 
aircraft  will  mean  that  information  in  front  of  the 
pilot  will  unfold  two  to  three  times  faster  than  in 
non-agile  aircraft.  Thus,  some  conventional 
symbology  (pitch  ladder  and  digital  readouts)  may 
change  too  rapidly  to  be  useful  to  the  pilot.  The  pilot 
has  to  “think  ahead”  more,  given  there  is  less  turning 
time  involved  in  getting  in  position  to  launch 
weapons.  These  agile  operations  require  decisions  to 
be  made  in  “microtime”  or  less  time  than  one 
typically  would  want  to  spend  weighing  options, 
making  decisions,  and  executing  actions.  The  ability 
of  the  system  to  provide  the  right  information  at  the 
right  time,  and  assist  the  pilot  in  determining  the 
right  course  of  actions,  is  the  crux  of  the  cockpit 
designers’  challenge. 

6.2.4.  NEED  FOR  TAILORING 

With  increasing  onboard  processing  capabilities, 
agile  aircraft  will  have  a concomitant  increase  in  the 
number  of  systems  and  possible  data  views.  The 
pilot’s  time  can  be  consumed  just  programming  the 
numerous  options  available.  Cockpit  design  and 
standard  operating  procedures  should  focus  solely  on 
those  options  essential  to  mission  requirements.  One 
mechanism  is  to  have  one  command  or  input 
automatically  activate  the  systems  and  set  up  the 
tasks  relevant  to  the  current  flight  segment  (e.g.,  air- 
to-air  versus  landing).  Only  those  options  required 
for  that  flight  segment  would  be  readily  accessible. 

6.2.5.  TUNNELLING  OF  ATTENTION 

It  cannot  be  assumed  that  pilots  will  scan  all 
available  information  sources  in  a timely  manner. 
Presenting  information  on  head  up  displays,  together 
with  the  demanding  agile  aircraft  mission,  can  result 
in  a tunneling  or  channeling  of  the  pilot’s  attention 
such  that  vital  head  down  information  is  missed.  It  is 
also  possible  for  a situation  (e.g.,  changing  threat 
scenario)  diat  attracts  the  pilot’s  attention  to  head 
down  displays  and  delays  the  pilot  from  returning  to 
a head  up  posture.  Therefore,  some  cueing 
mechanism  is  required  to  inform  pilots  of  critical 
information  or  a change  in  aircraft  or  mission  state 
that  needs  attention. 

6.2.6.  ENERGY  MANAGEMENT 

Given  the  complex  maneuvers  possible  with  agile 
aircraft,  anticipated  use  of  carefree  handling,  and 
decrease  in  sensory  feedback  (noise  and  buffet),  the 
pilot  needs  to  have  precise  timing  and  perception  of 
any  change  in  the  aircraft’s  energy  state.  Moreover, 
tire  pilot  needs  information  pertinent  to  energy 
management  to  weigh  the  advantages  of  different 
maneuvers  that  can  be  employed.  For  instance,  the 
pilot  needs  information  on  the  goodness  of  the 
launch  condition  to  assess  the  tactical  situation  and 
determine  whether  to  accept  a low  confidence  launch 
or  maneuver  to  a more  favorable  launch  position. 
This  is  especially  important  since  using  the 


advantages  of  an  agile  airframe  to  point  the  nose  at  a 
target  may  leave  the  aircraft  too  slow  to  recover 
speed  quickly  for  a missile  defense  maneuver. 

6.3.  HEAD  UP  CONTROLS/DISPLAYS 

Providing  and  controlling  information  “head  up” 
maximizes  the  amount  of  time  the  pilot  spends 
looking  out  the  canopy  for  threats.  To  date,  this 
advantage  is  primarily  realized  with  a head  up 
control  concept  and  a head  up  display  (HUD).  Head 
up  control  is  achieved  with  the  pilot’s  inceptors 
which  can  be  operated  with  tire  head  up.  Agile 
airframes  make  a sophisticated  system  that  integrates 
flight  and  propulsion  control  a definite  requirement. 
With  such  a carefree  handling  system,  the  stick  and 
throttle  can  be  used  to  maneuver  the  aircraft  inside 
the  whole  flight  envelope,  automatically  taking  into 
account  aircraft  limitations.  Head  up  control  is  also 
facilitated  with  additional  switches  located  on  the 
flight  controls;  this  hands-on-throttlc-and-stick 
(HOTAS)  concept  enables  selection  of  many  sensor, 
navigation,  and  weapon  systems  without  redirection 
of  the  pilot’s  gaze  point. 

A HUD  presents  symbology  projected  onto  a 
transparent  combiner.  Some  information,  such  as  a 
pitch  ladder  that  relates  directly  to  the  world,  can  be 
seen  superimposed  on  the  real  scene  to  facilitate 
display  interpretation.  The  display  can  also  relay  a 
sensor  image,  providing  a view  of  the  scene  ahead  at 
night  or  bad  weather.  Because  the  HUD  combiner  is 
fixed  to  the  top  of  the  instrument  panel,  the  pilot 
must  look  forward  along  the  aircraft  longitudinal  axis 
to  see  the  symbology.  Moreover,  targets  often  lie 
outside  its  limited  ficld-of-view. 

Helmet  mounted  displays  (HMDs)  have  been 
developed  as  one  means  of  extending  the  advantages 
of  the  head  up  transparent  display  concept  and 
overcoming  limitations  of  current  HUDs.  An  HMD 
can  provide  a wider  area  of  visual  information. 
Moreover,  with  a HMD,  displayed  information  is 
within  the  pilot’s  field-of-view  regardless  of  head 
movement  and  orientation.  Because  of  their  utility 
when  the  pilot  looks  both  along  and  away  from  the 
fore-aft  axis  of  the  aircraft,  HMDs  arc  predicted  to 
eventually  eliminate  the  need  for  HUDs. 

When  implemented  with  a hcad/hclinet  position 
tracker,  a HMD  system  can  also  provide  target 
cueing  and  sensor  guidance.  In  addition,  these 
Helmet  Mounted  Display/Tracker  (HMD/T)  Systems 
have  tremendous  capability  compared  to  earlier 
Helmet  Mounted  Sights  (HMS)  that  combined  a 
tracker  with  a sighting  reticle  to  provide  a simple 
aiming  mark  to  pilots.  HMD/T  systems,  along  with 
other  “head  up”  control  and  display  devices  (e.g., 
HOTAS  and  auditor}'  systems),  enable  pilots  to  focus 
attention  out  the  window  and  minimize  manual 
control  and  head  down  glances  which  can  cause 
disorientation  and/or  vertigo,  especially  in  extreme 
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+/-G.  This  is  even  more  critical  for  agile  aircraft  to 
support  maneuvering,  weapons  launch,  and 
evasion/survival. 

The  following  describes  candidate  head  up  controls 
and  displays.  This  presentation  will  focus  on  pilot 
usage  of  these  devices  for  agile  aircraft  applications, 
rather  than  on  the  mechanics  of  each  technology. 

6.3.1.  HELMET  MOUNTED  DISPLAY/ 
TRACKER  SYSTEMS  (HMD/T) 

Candidate  HMD/T  systems  have  three  major 
components:  1)  a head  or  helmet  mounted  visual 
display,  visually  directed,  2)  a means  of  tracking 
head  pointing  direction  (based  on  tire  assumption  that 
the  pilot  is  looking  in  the  general  direction  that  the 
head  is  pointing),  and  3)  a source  of  visual 
information  which  is  dependent  on  the  head  viewing 
direction  [2],  Information  displayed  on  the  HMD 
can  be  referenced  to  head  axes,  aircraft  axes,  earth 
axes,  or  any  combination  of  these  three.  Advances  in 
several  display  technologies  (miniature  cathode  ray 
tubes,  etc.),  make  HMDs  a definite  candidate  for 
agile  airframes  [3].  With  further  development,  other 
hardware  may  provide  additional  advantages  over 
conventional  approaches.  For  instance,  the  Virtual 
Retinal  Display™  scans  a lower  power  beam  of  light 
to  “paint”  rows  of  pixels  onto  the  retina  of  the  eye, 
creating  a high  resolution,  full  motion  image  without 
the  use  of  electronic  screens  [4], 

Tire  concept  of  HMD/T  operation  (Figure  6.1)  is  as 
follows:  the  pilot  looks  in  a particular  direction,  the 
head  tracker  determines  what  the  direction  is,  and  tire 
visual  information  source  produces  appropriate 
imagery  to  be  viewed  on  tire  display  by  the  pilot. 
The  direction  of  tire  head  can  also  be  used  as  a 
control  signal  for  a variety  of  aircraft  systems,  in 
addition  to  controlling  what  information  is  displayed. 
Thus,  the  HMD/T  system  serves  as  both  a head  up 
control  and  display,  with  an  instantaneous  field-of- 
view  around  25-40  degrees  subtended  visual  angle. 


Figure  6. 1 . Schematic  of  Helmet  Mounted 
Display/Tracker  System  Concept. 

(Reprinted  with  permission  from  Helmet-Mounted 
Displays  and  Sights,  by  Mordekhai  Velger.  Artech  House, 
lire.,  Norwood,  MA  USA.  wwvv.artechhouse.com) 


With  a HMD,  the  pilot  has  a global  view  of 
information  through  tire  whole  range  of  head 
positions  [5],  Head  up,  visually  coupled  information 
will  assist  the  pilot  in  looking  out  of  the  cockpit  to 
maintain  situational  awareness  in  a highly  dynamic 
flight  environment.  For  instance,  research  has  shown 
that  an  off-boresight  HMD  enhances  the  pilot’s 
search  capability,  tracking  performance  and 
survivability  in  a simulated  low-level,  high-speed 
airborne  surveillance/reconnaissance  mission  [6]  and 
facilitates  high  angle  target  search  and  intercept 
during  a simulated  air-to-air  engagement  [7],  With  a 
HMD,  both  the  angle  and  duration  of  off-boresight 
visual  scanning  were  increased.  The  extent  to  which 
this  advantage  can  be  realized  depends  on  tire 
information  presented  on  the  HMD,  as  tire 
symbology  can  occlude  tire  outside  world  view. 

For  combat,  tire  combination  of  agile  aircraft  and 
HMD/T  systems  may  offer  important  tactical 
advantages  when  used  in  conjunction  with  guided 
missiles.  A tracker  determines  the  position  of  the 
pilot’s  head  as  tire  target  is  followed  through  the 
display  on  tire  helmet  visor.  The  tracker  relays 
critical  information  to  the  computer  that,  in  turn, 
communicates  the  location  of  tire  target  to  tire  missile 
system.  When  the  weapons  lock  on  the  target,  the 
pilot  receives  feedback  and  pulls  the  trigger  located 
on  tire  control  stick  to  fire  tire  missile.  This  scenario 
represents  a total  paradigm  shift  in  the  way  within- 
visual  range  air-to-air  combat  is  fought.  Tire  nose  of 
the  aircraft  is  no  longer  the  sighting  reference  for 
cueing  the  weapon,  but  rather  the  pilot’s  helmet.  As 
long  as  the  target  is  within  range  and  can  be  viewed 
by  the  pilot  through  tire  display  in  the  helmet  visor, 
the  relative  position  of  the  aircraft  to  the  enemy  is 
not  critical.  Since  a hostile  contact  averages  only  30 
seconds  to  2 minutes,  any  time  saved  by  not  needing 
to  repoint  the  aircraft  helps  give  a quicker  first  shot 
capability  to  pilots.  This  capability  also  facilitates 
engagement  of  multiple  adversaries.  Using  a 
HMD/T  system,  a pilot  can  designate  and  launch  a 
missile  or  lock  tire  radar  and  immediately  turn  to  the 
next  target,  designating  sequentially  several  targets 
within  seconds  without  having  to  reposition  tire 
aircraft  [5], 

Another  advantage  of  a HMD/T  system  is  the  ability 
to  designate  targets  and  hand  off  their  location  to 
other  sensors  and  tire  theater  communications 
system,  in  general.  For  example,  the  pilot  can  steer  a 
FLIR  system  mounted  on  a steerable  gimbal  in  tire 
nose  of  the  aircraft.  Likewise,  a threat  detected  by  a 
sensor  can  be  used  to  cue  the  pilot  by  showing 
directional  information  to  the  threat  location  on  tire 
HMD.  The  pilot  can  also  designate  a ground 
position  and  then  call  up  cues  to  reacquire  tire  target, 
should  tire  pilot  lose  sight  of  it  during  maneuvering 
[5]. 
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These  potential  tactical  advantages  were 
demonstrated  in  several  simulated  scenarios  by 
operational  F-15  pilots  employing  a HMD/T  system 
[8],  Tire  simulation  pilots  reported  that  the  HMD/T: 
made  it  easier  to  accomplish  vvithin-visual  range 
radar  acquisition  and  get  visual  sighting  of  acquired 
targets,  saved  time  in  attacks,  provided  helpful 
weapon  data  while  visually  tracking  a target,  added 
tactics  capability  by  easing  simultaneous  AIM-9  and 
AIM-7  attacks,  and  avoided  sacrificing  basic  fighter 
maneuvers  to  launch  an  AIM-9  or  perform  a full 
system  gun  attack.  The  ability  to  accomplish  a 
visual  missile  attack  without  sacrificing  positional 
advantage  was  viewed  a key  advantage  of  the 
HMD/T.  The  pilots  commented  that  the  HMD/T 
provided  as  many  improvements  to  air-to-air 
operations  as  weapons  computers  have  provided  to 
air-to-ground  operations.  There  was  also  a marked 
exchange  ratio  advantage  for  the  pilots  with  the 
HMD/T. 

6.3.1. 1 Visual  Illusions  with  HMD/T  Systems 
Certain  vision  conditions  (empty  field  myopia  and 
accommodation  convergence  micropsia)  can  be 
problematic  with  HMD/T  usage  [9],  For  example, 
even  if  symbology  is  presented  on  a HMD  focused  at 
infinity,  overlaying  the  sky,  some  individuals’  eyes 
will  tend  to  focus  two  feet  out  from  the  display. 
Problems  such  as  this  can  result  in  misjudgments  of 
sizes  and  distances  to  external  objects. 

6.3.1.2  HMD/T  Symbology  Sizc/Location 
Symbology  size  needs  to  be  optimized  for  the  HUD 
field-of-view  viewing  and  the  goal  of  minimizing 
obstruction  of  the  outside  view.  Plus,  the  resolution 
of  the  HMD  will  impact  the  size  and  legibility  of 
presented  text  and  symbols.  One  study  [10]  has 
shown  that  recognition  of  symbolic  aircraft  presented 
on  a collimated  display  deteriorated  with  increased 
eccentricity  (5,  9,  and  13  degrees).  Aircraft  in  the 
periphery  had  to  be  displayed  for  a longer  time  than 
targets  near  the  fixation  axis,  for  viewers  to  classify' 
them  successfully.  Response  latencies  were  also 
longer  in  the  lower  and  left  visual  fields. 

6.3.1.3  HMD/T  Symbology  Format 

Most  of  the  information  requirements  for  agile 
aircraft  are  the  same  as  non-agile  aircraft.  For  flight 
control,  pilots  need  to  know  where  the  aircraft  is 
actually  going,  rates  of  change,  energy  management, 
and  how  to  recover  to  straight  and  level  flight. 
Given  the  dynamic  and  expanded  weapon  envelopes 
realized  by  agile  weapons,  pilots  will  need  enhanced 
estimations  of  the  probability  of  detection  and  or 
launch  as  well  as  accurate  information  on  the  threat 
situation,  ownship  susceptibilities,  and  sensor  ranges. 
Information  is  also  needed  to  assess  avoidance 
maneuvers  and  use  of  decoys. 

Advances  in  display  technologies  make  it  possible  to 
present  pilots  with  formats  that  span  from  simple 
lines  and  symbols  to  high  fidelity,  geo-specific 
perspective  scenes.  Having  the  HMD  format  attempt 


to  duplicate  the  organization  and  content  of  the  real 
world  is  attractive  in  the  sense  that  the  pilot  would 
have  everything  needed  (Figure  6.2  [2]).  The  optical 
flow  of  objects  could  give  natural  cues  as  to  altitude, 
attitude,  and  airspeed.  For  conditions  in  which  view 
of  the  outside  scene  is  poor  or  absent,  such  a display 
can  provide  a “virtual  cockpit.”  [5], 


Figure  6.2.  Example  Candidate  HMD  Symbology. 

For  flights  where  the  outside  scene  is  visible,  it 
remains  to  be  determined  whether  providing  this 
abundant  amount  of  information  is  in  the  pilot’s  best 
interests.  Contrary  to  viewing  a HUD,  the 
symbology  remains  constantly  in  front  of  the  pilot’s 
eye. 

For  any  proposed  format,  systematic  evaluations  of 
candidate  symbology  sets  arc  required.  These 
evaluations  should  start  by  examining  features  of 
individual  symbology  elements.  Indeed,  experience 
has  shown  that  the  usefulness  of  elements  depends 
on  the  pilot’s  current  flight  segment  and  information 
needs.  For  example,  consider  the  use  of  digital,  tape, 
and  dial  indicators  of  flight  parameters.  Some 
research  has  indicated  that  analogue  displays  are 
easier  to  process  than  digital  displays  since  the 
analog  information  is  extracted  more  intuitively, 
maps  more  directly  on  the  response  system  (i.e., 
analog  control  inputs),  and  requires  few  mental 
transformations.  Moreover,  if  the  digits  (e.g., 
vertical  velocity)  are  changing  very  rapidly,  the 
blurry  readout  is  useless,  especially  when  the  pilot 
only  needs  to  have  a general  indication  of  the  rate 
and  extent  of  change.  It  may  be  the  case  that  a dial 
format  (e.g.,  with  arc  scribing  the  outside  of  the 
altitude  dial  relative  to  changes  in  vertical  velocity) 
is  assimilated  more  easily  than  vertical  formats  [11]; 
however,  initial  results  using  a HMD  presentation 
failed  to  support  this  notion  [12],  With  a perspective 
format,  quantitative  estimates  are  even  more  difficult 
to  discern  and  adding  sealed  reference  marks  as  a 
remedy  tends  to  defeat  the  objective  of  providing  the 
pilot  the  impression  of  flying  into  the  perspective 
scene  [13], 
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Besides  the  requirement  to  evaluate  how 
conventional  flight  information  should  be  presented 
for  agile  aircraft  operation,  the  unique  information 
needs  for  these  missions  also  needs  to  be  considered. 
For  instance,  pilots  will  need  to  monitor  the  more 
extreme  angles-of-attack  that  can  be  achieved  by 
agile  aircraft.  HMDs  will  help  keep  the  flight  path 
vector  within  view,  except  when  the  aircraft  is  at  an 
extreme  angle  (e.g.,  60  degrees).  New  symbology 
might  be  useful.  One  candidate  HMD  symbology'  set 
evaluated  for  the  X-3 1 utilized  two  triangles, 
superimposed  and  appearing  as  one  triangle  for  0-30 
degrees  of  angle-of-attack.  For  30-70  degrees  angle- 
of-attack,  one  triangle  stayed  fixed  and  the  second 
grew  to  match  a point  on  a scale  inside  the  attitude 
reference  symbology  [14], 

There  have  also  been  several  evaluations  to 
determine  what  symbology  helps  exploit  the 
advantages  affording  by  using  a combination  of 
HMD/T  devices  and  precision  weapons.  One 
experiment  investigated  how  the  target  location 
information  should  be  related  to  the  pilot  [15],  Three 
symbology  orientations  were  evaluated.  In  one,  the 
symbology  was  relative  to  the  nose  of  the  aircraft, 
indicating  the  most  efficient  pursuit  vector  between 
the  ownship  and  an  airborne  target  location.  A 
second  orientation  referenced  head  movement, 
indicating  the  most  efficient  line  between  the  pilot’s 
line-of-sight  and  a target.  The  third  orientation 
evaluated  included  symbology  that  simultaneously 
presented  ownship  and  head  information.  The 
results  indicated  that  the  ownship  coordinate 
information  may  have  more  merit  than  traditionally 
believed  and  that  pilots  favored  the  combination 
which  presented  both  “look-to”  and  “fly-to”  locator 
lines  when  the  target  was  outside  of  the  HMD  field- 
of-view. 


reference,  as  well  as  an  artificial  horizon  line  and 
pitch  bars. 

The  German-developed  ASAR  (“orange  peel”) 
includes  a fixed  climb-dive  symbol  that  represents 
climb/dive  angle  by  its  relation  to  a half-circle  arc 
surrounding  the  symbol  [16],  The  upper  portion  of 
the  circle  is  invisible  during  straight  and  level  flight. 
The  visible  portion  of  the  circle  represents  the  area 
below  the  horizon  and  the  invisible  portion 
represents  the  area  above  the  horizon.  The  amount 
of  visible  orange  peel  translates  to  aircraft  pitch  (e.g., 
for  positive  pitch  attitudes  less  than  half  the  circle  is 
visible,  while  for  negative  pitch  attitudes  more  than 
half  is  visible).  As  the  climb  angle  increases,  the 
visible  negative  angle  area  of  the  arc  begins  to 
narrow  in  proportion  to  the  climb  angle.  With  an 
increase  in  dive  angle,  the  arc  closes  to  forms  a more 
complete  circle  (Figure  6.3).  At  a 90-degree  dive 
angle,  the  arc  forms  nearly  a complete  circle,  leaving 
a small  gap  to  cue  the  pilot  of  the  most  efficient 
direction  to  recover  from  the  dive.  During  a roll,  the 
arc  rotates  about  the  climb-dive  symbol. 


Heading  Indicator 


34 


I 

35  00 

I 


Airspeed  - 


Aim-Sight  Reticle  ' 


Climb-Dive  Symbol 


X 


Vertical  Velocity 
\ 


-•/  •' 
\ \ 


Dial  Altimeter 
with  Digital 
Information 


^ Artificial  Horizon 
" ASAR  Format 


Consideration  of  HMD  symbology  with  respect  to 
weapons  needs  to  consider  the  exact  flight  mission 
anticipated,  as  information  needs  may  differ 
depending  on  whether  the  pilot  is  engaged  in  air-to- 
air  combat,  air-to-ground  attacks,  or  missile  evasion 
(in  addition  to  navigation  and  landing  piloting  tasks). 
The  designer’s  objective  is  to  provide  the 
information  required  for  each  flight  segment,  yet 
minimize  the  pilots’  training  burden  by  keeping 
symbology  sets  as  similar  as  possible.  For  instance, 
there  have  been  several  studies  addressing  how  and 
when  ownship  information  should  be  presented.  One 
experiment  [7]  examined  if  ownship  status 

information  within  the  HMD  symbology  set  is 
necessary  for  air-to-air  applications.  Several 
ownship  status  formats  were  evaluated,  including  die 
Standard  AtUtude  Reference,  the  Arc  Segmented 
Attitude  Reference  (ASAR)  and  the  Theta 
Attitude/Direction  Indicator  (Theta).  In  die  standard 
format  developed  by  the  US  Air  Force,  the  attitude 
set  includes  a helmet  fixed  inverted  “T”  climb/dive 
symbol  oriented  as  an  inside-out  flight  path 
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Figure  6.3.  Example  ASAR  (top)  and  Theta 
(bottom)  HMD  Symbology  [12]. 


Anodier  format  featured  a Theta  Attitude/Direction 
Indicator  [16]  developed  at  the  Air  Force  Research 
Laboratory  (Figure  6.3).  The  symbology  integrates 
heading  information  and  attitude  symbology7  with  a 
simulated  three-dimensional,  transparent,  wire-frame 
half-ball  consisting  of  arced  lines.  The  longitudinal 
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lines  serve  as  an  azimuth  (heading)  position 
reference  in  45-degree  increments.  The  ball  is  free 
to  rotate  about  all  three  of  its  axes  to  represent 
rotation  of  the  aircraft  in  those  axes.  Continuous 
lines  on  the  upper  portion  and  segmented  lines  on  the 
lower  portion  represent  climb  and  dive  areas, 
respectively.  Within  the  ball  there  is  a climb/dive 
symbol  and  the  cardinal  headings  are  marked  by 
letters.  The  format  is  mechanized  like  a standard, 
three -axis  attitude  direction  indicator  ball.  The 
results  of  this  experiment  failed  to  show  any 
interpretation  or  usability  differences  among  the 
formats  used  to  present  ownship  information. 
However,  the  pilots  definitely  preferred  that  ownship 
status  be  included  in  the  HMD  symbology  set  [7], 

Another  study  specifically  examined  the  ASAR  and 
Theta  symbology  suites  for  a HMD  used  in  the  X-3 1 
[17],  The  pilots  commented  that  for  close-in- 
combat, attitude  symbology  was  less  critical,  because 
the  pilot  flies  relative  to  the  opponent  aircraft.  Pilots 
found  the  ASAR  useful  as  a large  amplitude  pitch 
reference  for  locating  the  horizon  or  recovering  from 
an  unusual  attitude.  This  symbology  is  very 
compelling  and  effective  for  simple,  instant 
instruction.  However,  they  questioned  its  utility  as  a 
precision  instrument.  The  Theta  symbology  was 
found  easy  to  interpret  and  was  the  preferred  attitude 
reference.  The  globe  provided  a good  analysis  of  the 
situation.  However,  its  utility  in  complex  scenarios 
remains  to  be  determined. 

In  a more  recent  effort  at  the  Air  Force  Research 
Laboratory,  a “non-distributed  flight  reference 
symbology”  was  designed  to  supplement  HMD 
target  acquisition  information  with  ownship  status 
information,  the  latter  particularly  useful  during  high 
off-boresight  targeting  tasks  [18],  The  key  challenge 
was  to  ensure  the  presented  information  is  useful 
without  any  associated  clutter  or  disorientation 
incurred  by  its  presence.  This  non-distributed  flight 
reference  symbol  set  presents  ownship  aircraft 
reference  information  close  together  and  positioned 
within  the  attitude  symbology  (sec  Figure  6.4).  The 
primary  flight  information  is  spatially  arranged  so 
that  tire  conventional  basic  “T”  layout  is  maintained 
with  airspeed  to  the  left  of  altitude  and  heading 
between  airspeed  and  altitude.  The  information  is 
presented  digitally  inside  an  outline  designed  to 
mimic  the  shape  of  aircraft  wings  and  tail. 
Collectively,  this  compact  information  montage  can 
be  located  anywhere  in  the  HMD  ficld-of-view  (e  g., 
near  the  bottom  during  air-to-air  applications). 

The  aircraft  symbol  is  fixed  relative  to  the  HMD 
field-of-view  and  the  attitude  symbology'  moves 
about  it.  The  flight  path  angle  and  roll  of  the 
ownship  montage  is  represented  by  its  relation  to  a 
half  circle  arc  (using  the  ASAR  approach  described 
earlier).  Heading  tags  appear  at  extreme  climb  and 
dive  angle  to  give  additional  indication  of  ownship 


roll.  This  functionality  was  found  useful  in  previous 
evaluations  of  the  Theta  attitude  reference 
symbology  and  helps  provide  orientation  information 
throughout  the  full  aircraft-maneuvering  envelope. 
During  rolling  maneuvers,  the  arc  and  artificial 
horizon  rotate  about  the  ownship  symbol. 

It  is  only  through  systematic  evaluation  of  candidate 
symbology’  sets  for  specific  flight  tasks  that  optimal 
HMD  symbology  can  be  identified.  Unfortunately,  a 
format  found  to  quickly  provide  pilots  with  an 
overall  situational  awareness  and  orientation 
perspective,  may  not  provide  the  information 
required  to  precisely  control  the  aircraft  through  a 
commanded  mission.  Also,  the  ideal  format  may 
depend  on  environmental  factors,  ground  detail,  and 
the  availability  of  an  outside  reference.  For  instance 
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Figure  6.4.  Non-Distributed  Flight  Reference 
Candidate  Symbol  Set  with  modified  ASAR  Attitude 
Symbology'  during  various  Climb  and  Dive  Angles 
and  Roll  Maneuvering  [18], 
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the  reference  frame  used  to  present  attitude 
information  to  the  pilot  (“pilot  perspective”)  needs  to 
be  determined.  Two  major  display  concepts  are 
commonly  referred  to  as  inside-out  or  “pilot’s  view” 
versus  outside-in  or  “God’s  view.”  Typically, 
inside-out  displays  are  viewed  more  appropriate  for 
precise  flight  control  and  outside-in  displays  more 
appropriate  for  navigation  and  landing  tasks. 
However,  some  research  has  demonstrated  that  an 
outside-in  format  is  superior  for  unusual-attitude 
recovery  [19].  In  fact,  pilots  have  noted  that 
traditional  aircraft-referenced  inside-out  attitude 
displays  are  more  difficult  to  interpret  when  the  head 
is  moved  off-axis. 

Multiple  coordinate  reference  frames  can  also  be 
used.  For  example,  one  complex  format  envisioned 
for  a HMD  has  a large  instantaneous  field-of-view 
and  multiple  cueing  symbols.  One  aiming  symbol 
would  be  in  the  upper  portion  of  the  HMD  for 
designating  aiming  points  outside  the  aircraft  and 
another  aiming  symbol  would  be  in  the  lower  portion 
of  the  display  to  designate  space  stabilized  electronic 
cockpit  switches  and  functions  [2].  The  position  of 
the  cockpit  switches  imaged  on  the  HMD  stays  fixed 
with  respect  to  the  cockpit,  while  the  lower  reticle 
moves  with  changing  head  position.  When  the  lower 
reticle  is  placed  over  the  electronic  cockpit  switch, 
it’s  visual  form  changes  to  indicate  it  is  active  and  is 
being  designated  by  head  position.  The  pilot  must 
then  give  a consent  response  by  activating  a single 
standard  switch  located  on  one  of  the  primary 
manual  controllers  (e.g.,  joystick)  or  by  issuing  a 
standardized  verbal  command.  Meanwhile,  the 
upper  aiming  reticle  remains  active  for  designating 
outside  world  targets  through  other  electronic  control 
loops.  Although  it  appears  that  this  envisioned 
format  would  provide  pilots  with  enormous  display 
and  control  capabilities,  there  are  numerous 
ergonomics  issues  that  need  to  be  examined  before 
these  anticipated  advantages  can  be  realized. 

6.3.1.4  Pictorial  Portrayal  of  Information  in 
HMD/T  Systems 

Symbols  and  alphanumerics  presented  in  display 
formats  depicting  status  of  aircraft  systems  can  be 
viewed  as  individual  chunks  of  information  that  must 
be  perceived  and  cognitively  integrated  by  the  pilot. 
Humans  are  limited  as  to  the  number  of  information 
chunks  that  can  be  managed  at  a time.  Advances  in 
the  generation  of  display  graphics  enable  pictorial 
portrayal  of  information  that  groups  individual 
pieces  of  information  into  fewer  chunks. 
Theoretically,  this  reduces  the  pilot’s  workload 
because  the  processing  required  to  chunk  the 
information  has  already  been  accomplished  [20]. 
The  pilot  can  more  rapidly  acquire  the  message 
(assuming  the  pictorial  is  easy  to  interpret)  and  then 
devote  time  executing  a response. 

For  HMD  formats,  the  most  popular  pictorial 
presentation  entertained  is  a three-dimensional 


perspective  path  to  assist  the  pilot  in  flight  control 
(see  below).  Pictorial  formats  occupy  more  display 
area  than  conventional  formats.  Given  the  limited 
field-of-view  of  the  HMD  and  a desire  to  minimize 
the  extent  to  which  symbology  interferes  with  the 
pilot’s  outside  view,  the  added  value  of  pictorial 
formats  for  HMDs  needs  to  be  verified. 

6.3.1.5  Stereopsis  Cues  in  HMD/T  Systems 
The  introduction  of  true  depth  cues  via  stereopsis 
techniques  in  HMDs  offers  a means  of  further 
enhancing  pictorial  displays,  particularly  in 
improving  the  perception  of  pictorial  layouts.  For 
example,  in  one  format  a range  marker  element 
(waterline  symbol)  provides  a non-stereo  cue  in  that 
when  the  lead  aircraft  is  at  the  desired  range,  the 
wingspan  of  the  aircraft  symbol  is  the  same  width  as 
the  ownship  symbol  (the  desired  range  marker). 
Inclusion  of  stereo  depth  cues  with  this  symbology 
was  found  to  improve  performance  by  18%  in  a 
simulation  evaluation  [21], 

Probably  the  most  entertained  cockpit  application  of 
this  technology,  is  to  provide  the  pilot  with  a three- 
dimensional  “pathway-in-the-sky”  which  integrates 
all  relevant  information  into  a single  display  and  the 
pilot’s  task  is  reduced  to  simply  following  the  path 
(Figure  6.5).  With  the  current  accuracy  afforded  by 


Figure  6.5.  Examples  of  Pictorial  Pathway-in-the- 
Sky  Flight  Display  Formats  [22,23], 
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Global  Positioning  and  vast  digital  terrain  data  bases, 
the  pathway-in-tlve-sky  visualization  concept  is  even 
more  obtainable.  In  the  example  flight  path  display 
formats  depicted  in  Figure  6.5,  the  pilot’s  task 
requires  guiding  the  aircraft  through  the  center  of  the 
channel  to  accurately  stay  on  course.  Each  of  the 
channels  extends  into  the  distance  so  the  upcoming 
changes  in  the  path  can  be  anticipated.  In  the 
channel  depicted  at  the  bottom  of  Figure  6.5,  the 
aircraft  is  a stationary  symbol  and  the  channel  moves 
about  it  with  changes  in  lateral  and  vertical  direction. 
This  channel  configuration  depicts  the  aircraft  to  be 
slightly  to  the  right  of  command  path,  but  flying  the 
command  altitude. 

Of  course  during  combat,  the  dynamics  of  the 
maneuvers  make  using  a path-type  format  as  a 
command  display  less  appropriate.  However,  such  a 
display  could  be  very  useful  for:  1)  providing 
predictive  information  to  the  pilot  - illustrating  the 
endpoint  if  the  aircraft  continues  in  its  current  flight 
path,  and  2)  providing  short  term  command 
information  to  execute  a pilot-selected  stereotypical 
maneuver  (e.g.,  scissors  maneuver).  In  considering 
these  channel  displays  for  agile  aircraft,  there  is  a 
body  of  relevant  research.  Researchers  have  found 
that  three-dimensional  tunnels  where  the  viewpoint 
of  the  display  corresponds  to  the  position  of  the  pilot 
(i.e.,  fully  egocentric)  arc  superior  for  flight  path 
control,  particularly  for  flying  curved  paths  [13,  23], 
Subjects  found  the  channel  display  intuitive  and  to 
provide  quick  and  simple  orientation.  Integrating 
information  pertaining  to  the  aircraft’s  vertical 
situation,  horizontal  situation  and  profile  situation 
also  relieves  the  pilot  from  scanning  multiple 
displays  to  acquire  the  same  information.  However, 
such  a display  reduces  situational  awareness  due  to 
its  narrow  field-of-view,  making  it  difficult  for  the 
pilot  to  be  aware  of  hazards  in  the  surrounding 
airspace.  If  the  field-of-view  is  increased,  however, 
valuable  display  real  estate  is  consumed  or  a 
distortion  occurs  as  real  space  is  compressed.  This, 
in  turn,  also  disrupts  situational  awareness, 
increasing  the  ambiguity  of  where  things  arc  in 
space.  The  resolution  of  predictor  information  in  the 
display  is  already  less,  since  the  perspective 
presentation  means  a reduction  of  size  for  objects  far 
away.  Moreover,  it  is  difficult  to  get  sufficient 
quantitative  information,  unless  additional  scaled 
reference  markers  and  readouts  are  added.  These 
items,  though,  lessen  the  natural  impression  of  flying 
through  the  channel  [13].  In  an  experiment 
evaluating  a three-dimensional  perspective  flight 
display  compared  to  multiple  two-dimensional  planar 
displays,  measures  for  flight  performance  and 
situational  awareness  were  worse  for  the  perspective 
display.  The  subjects  commented  that  the  key 
problem  was  the  ambiguity  in  depth  judgement  along 
the  line-of-sight  that  the  perspective  display  caused 
when  the  aircraft  was  approaching  landmarks  [24], 


A more  near  term  application  of  stereopsis  cueing  for 
agile  aircraft  is  to  present  different  categories  or 
classes  of  information  at  different  levels  in  depth 
[25],  With  this  application,  only  a few  levels  in 
depth  are  needed  and  there  is  no  requirement  that 
they  accurately  represent  a certain  dimension  in 
depth.  Use  of  three-dimensional  presentations  on 
HMDs  helps  dcclutter  information  and  enables  the 
pilot  to  more  efficiently  switch  attention  between 
different  information  classes.  For  instance,  the 
altitude,  heading  and  airspeed  indicators  could 
appear  on  a different  plane  from  the  aircraft  symbol 
[25],  As  one  pilot  interviewed  by  the  Working 
Group  commented,  if  information  can  be  positioned 
at  different  depth  levels,  much  more  information  can 
be  presented  on  the  HMD.  Of  course,  research 
would  be  required  to  detennine  the  optimal 
assignment  of  information  to  depth  levels  - whether 
by  information  class  (flight,  weapon  status,  threat 
status,  etc.),  priority',  flight  segment,  or  some  other 
combination.  Another  possibility  is  to  have  the 
coding  pilot  selectable. 

6.3.1.6  Use  of  Color  in  HMD/T  Systems 
Assuming  that  the  image  source  for  a HMD  is  such 
to  make  colors  visible  under  high  ambient 
illumination  conditions,  use  of  color  in  display 
formats  with  discrete  elements  can  make  information 
uptake  easier  and  faster.  A likely  near  term 
application  of  color  is  in  the  pitch  ladder  symbology. 
Monochrome  coding  has  already  been  applied  in 
conventional  formats  to  help  reduce  ambiguity 
between  the  positive  and  negative  pitch  bars  and 
facilitate  recovery  from  unusual  attitudes.  This  has 
included  using  different  shape  coding  for  negative 
(bendy)  and  positive  (tapered)  bars.  Color  coding 
flic  pitch  ladder  (positive  bars  blue,  negative  bars 
brown)  has  been  found  to  be  beneficial  in 
simulations,  especially  when  used  in  conjunction 
with  shape  coding  [26], 

Application  of  color  (red  and  green)  to  help  code 
target  location,  tracking,  and  weapons  deployment 
was  examined  in  a simulation  of  off-borcsight 
weapon  aiming  [27],  Overall,  the  pilots  preferred  the 
color-coded  symbology  to  the  monochrome  baseline. 
Furthermore,  a “red  means  shoot”  color-coding 
strategy  (involving  a progression  from  green  to  red 
as  an  indication  of  shoot-criteria  satisfaction)  was 
preferred  over  a “green  means  go”  strategy 
(progression  from  red  to  green).  In  a subsequent 
study,  the  “red  means  shoot”  coding  was 
systematically  compared  to  a monochrome  baseline 
HMD  symbology  in  an  air-to-air  simulated  weapon 
delivery  scenario.  Results  showed  the  “red  means 
shoot”  symbology'  produced  significantly  faster  shots 
without  degrading  the  probability  of  kill  [28], 

Any  application  of  color  in  agile  aircraft  should  be 
aware  of  flic  results  of  an  initial  evaluation  showing 
sequential  changes  in  color  perception  during 
relaxed,  gradual  onset  of  Gz  acceleration  [29],  The 
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effects  occurred  at  4 Gz  and  the  first  hue  shift  was  a 
disappearance  of  light  blue  into  white,  indicating  that 
use  of  light  blue  on  a light  background  may  be 
problematic  during  a high  Gz  turn.  The  second  shift 
was  green  to  yellow.  Thus,  use  of  green  and  yellow 
to  code  classes  of  threats  on  a display  may  be 
confusing.  Such  findings  support  formats  that 
employ  redundant  (e.g.,  color  and  shape)  coding. 

6,3.2.  EYE-BASED  CONTROL 

The  next  plausible  extension  of  the  capabilities  of 
HMD/T  systems  is  the  integration  of  eye  tracking  to 
enable  control  of  crew  station  functions  using  the 
pilot’s  eye  line-of-sight  [30],  In  that  the  visual 
system  is  the  primary  channel  for  acquiring 
information  and  eye  muscles  are  extremely  fast  and 
respond  very  quickly,  it  is  advantageous  to  have  the 
direction  of  eye  gaze  serve  as  a control  input.  In 
other  words,  if  the  pilot  is  looking  at  a target,  it  is 
more  efficient  to  use  the  pilot’s  gaze  to  aim  a 
weapon,  rather  than  align  the  head  or  manually  slew 
a displayed  cursor  over  the  target.  In  this  manner, 
eye-based  control  can  increase  the  envelope  and 
speed  of  target  acquisition  with  a HMD/T  system. 
Line-of-sight  cueing  between  pilots  can  also  be 
facilitated  by  eye  designation  of  points  of  interests. 
The  pilots  briefing  the  Working  Group  were  very 
positive  as  to  the  increased  capability  that  could  be 
realized  with  eye-based  control.  Moreover,  eye 
motion  is  more  feasible  under  high  acceleration 
conditions,  compared  to  head  or  hand  movement. 

For  eye-based  control  to  be  useful,  however,  it  is 
important  that  the  pilot’s  eye  movements  remain 
natural  and  not  involve  unusual  blinking  or  lengthy 
fixations.  Eye-based  control  is  similar  to  operating  a 
computer  mouse  in  the  sense  that  gaze  position 
indicates  the  position  or  response  option  on  a 
display,  and  some  method  analogous  to  a mouse 
button  press  is  used  to  trigger  the  response.  Without 
the  additional  consent  response,  a “Midas  touch” 
problem  could  occur,  with  commands  activating 
wherever  the  pilot  looks.  Different  types  of  consent 
responses  have  been  evaluated  [31]  and  it  is 
recommended  that  a dedicated,  conveniently  located 
button  (e.g.,  on  the  joystick)  be  employed  as  a 
universal  consent  response.  If  the  pilot’s  gaze  is 
only  being  utilized  to  call  up  additional  data  for  eye 
designated  icons,  then  perhaps  only  a short  fixation 
is  sufficient,  without  a consent  response.  In  this 
manner,  the  pilot’s  sequential  review  of  a series  of 
targets  can  be  made  more  rapidly,  with  detailed 
information  popping  up,  as  the  gaze  briefly  pauses 
on  each  target. 

Although  current  eye  tracking  systems  are  not  flight 
worthy  for  agile  aircraft  applications,  numerous 
efforts  are  underway  to  explore  how  eye  tracking 
optics  might  be  integrated  into  HMD  systems  and 
how  best  to  track  the  eye  under  varying  illumination 
conditions.  It  is  anticipated  that  eye-based  control 


will  eventually  be  feasible  for  agile  aircraft  and  be 
used  to  designate  display  areas  subtending 
approximately  1 degree  of  visual  angle  (by  fixating 
50-100  milliseconds)  [32],  Designation  of  very 
small  targets  may  be  problematic;  however,  there  are 
several  techniques  the  designer  can  employ  to  aid  in 
the  gaze-based  selection  of  densely  packed  targets 
[33],  Improvement  in  eye  tracking  technology  will 
also  be  required  to  enable  eye-based  control  at  more 
extreme  “look  angles”  (e.g.,  +40  degrees  azimuth 
and  elevation). 

6.3.3.  ELECTROMYOGRAPHIC  (EMG)- 
BASED  CONTROL 

It  is  feasible  to  modify  the  hardware  or  helmet 
housing  a HMD/T  system,  or  the  pilot’s  oxygen 
mask,  to  position  electrodes  on  the  surface  of  the 
skin  which  detect  the  asynchronous  firing  of 
hundreds  of  groups  of  muscle  fibers.  These 
electrical  signals  that  accompany  muscle 
contractions,  rather  than  the  movement  produced  by 
these  contractions,  can  be  used  to  provide  EMG- 
based  control.  Most  commonly,  these  electrical 
signals  are  compared  to  some  threshold  value  to 
derive  a binary  control  input  - above  threshold 
initiates  one  control  action,  below  threshold  initiates 
another  [34],  Development  is  still  required  to 
optimize  the  signals  employed,  assess  the  stability  of 
electrode  contact  over  time,  and  minimize  the  effect 
of  operator  movement  and  external  electrical  activity 
on  signal  recordings.  However,  EMG-based  control 
is  a far-term  candidate  head  up  controller  that 
enables  the  pilot  to  make  discrete  responses  without 
using  the  hands.  To  implement  EMG-based  control, 
it  is  important  to  choose  a body  movement  that  does 
not  interfere  with  the  pilot’s  normal  functions,  is  not 
likely  to  be  made  during  normal  activity  or  in 
response  to  acceleration  loading,  and  can  be 
implemented  such  that  the  system  can  discriminate  a 
purposeful  EMG  input  from  an  inadvertent  one.  To 
date,  subtle/slight  eyebrow  lifts  and  jaw  clenches 
have  been  successfully  used  in  concept 
demonstrations  as  enter  and  tab  functions  on  a 
computer  task.  However,  these  simulations  targeted 
ground-based  tasks  and  the  results  may  not  be 
applicable  to  agile  aircraft  controls. 

6.3.4.  ELECTROENCEPHALOGRAPHIC 
(EEG)-BASED  CONTROL 

Electrodes  integrated  into  the  pilot’s  headgear 
positioned  over  specific  areas  of  the  scalp  can 
provide  the  necessary  signals  to  implement  EEG- 
based  control  [35].  This  type  of  control  translates 
the  electrical  activity  of  the  brain  into  a control 
signal.  In  one  approach,  EEG  patterns  are  brought 
under  conscious  voluntary  control  with  training  and 
biofeedback.  A more  applicable  approach  harnesses 
naturally  occurring  brain  responses  to  modulated 
stimuli.  These  brain  responses  include  components 
that  modulate  at  the  same  frequency  as  the  evoking 
stimuli.  Selectable  items  of  a display  are  modulated 
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at  different  frequencies.  The  pilot’s  choice  (gaze 
point)  between  selectable  items  can  be  identified  by 
detecting  which  frequency  pattern  is  dominant  in  the 
visual  evoked  brain  activity.  In  effect,  the 
advantages  of  eye  gaze-based  control  can  be  realized 
with  less  expensive  and  obtrusive  components  with 
this  mechanization. 

Optimization  of  this  head  up  control  requires 
minimizing  the  time  required  for  signal  processing, 
developing  easily  donned  electrodes,  and  minimizing 
the  distraction  produced  by  modulating  (flashing) 
display  items.  Research  is  underway  to  investigate 
whether  the  brain  responses  produced  by  high- 
frequency  modulated  stimuli  (that  the  pilot  does  not 
perceive  as  flashing)  are  adequate  for  implementing 
EEG-based  control  [36], 

6.3.5.  SPEECH-BASED  CONTROL 

Speech  recognition  technology  allows  the  pilot’s 
speech  signals  to  be  used  to  carry  out  preset  activities 
(e.g.,  allocate  missiles  to  targets,  change  navigation 
route  and  radio  frequency,  alter  displays,  control 
radar,  etc.).  Unless  a high  recognition  reliability  can 
be  achieved  (e.g.,  95%  correct  recognition  under  4 
G),  voice  entries  may  need  an  additional  validation 
step  for  many  agile  aircraft  control  functions. 
Design  factors  that  influence  the  utility  of  speech 
recognizers  include:  acoustic  similarity  of 

commands,  length  of  words,  microphone  placement, 
consistency  of  the  speaker’s  speech,  vocabulary  size, 
and  the  extent  to  which  the  order  of  commands  is 
restricted  [1],  A key  challenge  to  the  application  of 
speech-based  control  for  agile  aircraft  is  efficient 
dialogue  design.  The  vocabulary  and  syntax  must  be 
manageable,  without  imposing  a great  memory  load 
or  interfering  with  communications.  If  the  pilot  has 
to  look  down  into  the  cockpit  to  read  command 
names  off  a menu,  then  the  head  up  control 
advantage  of  speech-based  control  is  compromised. 
Use  of  speech  input  also  has  the  potential  of  rapidly 
accessing  functions  several  levels  down  the 
hierarchical  structure  of  a multifunction  control.  On 
the  other  hand,  selection  of  a dedicated,  frequently 
selected  switch  (e.g.,  HOT  AS  concept)  may  be  more 
rapid  than  the  mental  processing  involved  in  issuing 
a verbal  command  and  the  time  required  by  the  voice 
recognizer  to  process  the  signal. 

There  are  several  environmental  factors,  which  can 
impact  the  performance  of  speech  systems:  high 
ambient  noise,  vibration,  stress  level  of  the  pilot,  and 
acceleration  (although,  “intelligible  speech”  can  be 
produced  up  to  9 G).  To  compensate  for  these  shifts 
in  speech  due  to  changes  in  the  environment, 
adaptation  algoritluns  are  required  in  the  speech 
processing,  as  well  as  noise  canceling  hardware  [37], 

6.3.6.  GESTURE-BASED  CONTROL 

Besides  using  the  electrical  activity  produced  by 
slight  facial  gestures,  other  small  sensors  mounted  in 


the  oxygen  mask  can  be  used  to  track  fine 
movements  of  the  pilot’s  face  or  lips.  Optical  and 
ultrasonic  sensing  technologies,  for  instance,  have 
been  used  to  monitor  an  operator’s  mouth 
movement.  In  one  implementation,  a headset  boom 
located  in  front  of  the  speaker’s  lips  contains  an 
ultrasonic  signal  transmitter  and  receiver.  A 
piezoelectric  material  and  a 40  KHz  oscillator  are 
used  to  create  a continuous  wave  ultrasonic  signal 
[38],  The  transmitted  signal  is  reflected  off  the 
speaker’s  mouth,  creating  a standing  wave  that 
changes  with  movements  in  the  speaker’s  lips.  The 
magnitude  of  the  received  signal  is  processed  to 
produce  a low  frequency  output  signal  that  can  be 
analyzed  to  produce  lip  motion  templates. 

There  are  two  candidate  applications  of  lip  motion 
measurement.  In  one,  the  pilot’s  lip  movements  are 
processed  during  speech  inputs  to  provide  “lip 
reading.”  An  experiment  using  an  ultrasonic  lip 
motion  detector  in  a speaker  dependent,  isolated 
word  recognition  task  demonstrated  that  the 
combination  of  ultrasonic  and  acoustic  recognizers 
enhances  speech  recognition  in  noisy  environments 
[38],  Alternatively,  symbolic  lip  gestures  can  be 
translated  into  communication  tokens  that  arc  used  as 
control  inputs. 

6.3.7.  DISPLAYS  IN  THE  PERIPHERAL 
VISUAL  FIELD 

Given  the  increased  likelihood  of  spatial 
disorientation  in  agile  aircraft,  the  “Malcolm 
Horizon”  attitude  display  was  reviewed  [39].  This 
concept  involves  projecting  an  artificial  bar  of  light 
across  the  instrument  panel  and  having  it  move  in  a 
manner  corresponding  to  the  horizon  outside  the 
aircraft.  Such  a display  enables  supplemental 
attitude  information  to  be  acquired  in  the  pilot’s 
periphery.  Although  pilot  response  to 
demonstrations  of  this  concept  was  generally 
positive,  problems  with  upright-inverted  ambiguity 
were  noted. 

The  display  of  information  in  the  pilot’s  periphery 
also  takes  advantage  of  the  human’s  increased  ability 
to  detect  movement  in  the  periphery,  compared  to 
central  vision.  Tlius  changes  in  attitude  may  be  more 
readily  detected  with  a peripheral  display.  On  the 
other  hand,  this  phenomenon  may  make  the  frequent 
detections  of  attitude  changes  a source  of  distraction. 
Or  the  peripheral  display  may  not  be  perceived  at  all, 
if  the  pilot  is  attending  to  the  central  field.  To  date, 
efforts  to  develop  large-scale  peripheral  attitude 
displays  have  met  with  only  mixed  success  and  their 
value  for  agile  aircraft  can  only  be  determined  with 
additional  evaluation.  These  evaluations  should 
consider  a more  textured  format  that  provides  a 
“flow  field”  in  the  periphery'  and  the  likely  utility  of 
HMD/T  systems.  Other  modalities  (tactile  and 
auditory)  may  also  be  useful  for  increasing  attitude 
awareness. 
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6.3.8.  TACTILE  DISPLAYS 

Tactile  displays  are  another  candidate  device  for 
agile  aircraft  applications.  (“Tactile  displays”, 
herein,  refer  to  devices  that  convey  distributed 
sensations,  rather  than  devices  that  provide  vector 
force  haptic  feedback.)  Tactile  displays  located  on 
the  human  trunk  have  the  potential  of  providing 
information  without  interfering  with  motor  or  any 
other  sensory  function.  Also,  they  are  “head  up” 
displays  since  the  perception  does  not  require  the 
pilot  to  glance  into  the  cockpit.  The  human  skin 
responds  to  several  distributed  physical  quantities: 
vibrations,  small-scale  shape  or  pressure 
distributions,  and  thermal  properties.  Vibration 
based  displays  use  frequencies  ranging  from  a few 
Hertz  (Hz)  to  a few  hundred  Hz.  For  aircraft 
applications,  the  distribution  of  skin  stimulation  is 
mapped  to  the  state  of  some  aircraft  parameter  or 
system.  For  instance,  one  university  group  [40]  is 
examining  microelectronic  mechanical  systems 
which,  when  integrated  with  a fabric  suit,  can 
provide  a thumping  or  gentle  pressure  on  a certain 
part  of  the  pilot’s  abdomen  to  notify  the  pilot  when 
the  aircraft  is  listing  to  one  side. 

One  tactile  display  has  already  been  demonstrated 
for  a helicopter  application.  The  Tactile  Situation 
Awareness  System  (TASS)  is  designed  to  provide  an 
indication  of  velocity  direction  and  velocity  vector 
magnitude  [41],  Specifically,  22  pneumatically 
driven  tactors  (1.25  in  diameter)  were  integrated  into 
an  F-22  cooling  vest  worn  on  pilot’s  torso  (Figure 
6.6).  By  activating  (vibrating  the  membrane  at  +/-2 
PSI  amplitude  at  50  Hz)  different  tactor  locations  on 
the  torso,  the  direction  of  helicopter  drift  (in  45 
degree  increments)  was  indicated  and  the  tactor 
activation  pulse  pattern  (rate  of  turning  tactor  on  and 
off)  was  used  to  indicate  the  magnitude  of  drift.  The 
preliminary  results  from  four  pilots  completing  hover 
maneuvers  suggest  that  such  a tactile  display  may 
improve  pilot  awareness  of  helicopter  movement  and 
reduce  workload,  especially  under  reduced  visibility 
conditions.  As  one  pilot  commented:  “I  could  feel 
the  tactors  before  I could  detect  visual  cues  of 
movement.”  This  promising  technology  may  also 


Figure  6.6.  Schematic  of  Tactile  Display  Concept 


have  additional  applications  for  altitude  awareness 
(e  g.,  using  a tactile  display  on  the  arm),  position 
maintenance  around  a reference  point,  directional 
indication  of  threats,  and  non-verbal  communication. 
The  utility  of  tactile  displays  is  determined  in  part  by 
their  limited  resolution  (discrete  number  of  tactors), 
the  limitations  in  the  rate  at  which  pilots’  can 
effectively  use  incoming  tactile  data,  and  their  utility 
under  acceleration.  In  particular,  evaluation  is 
required  to  determine  how  pilots  resolve  any 
conflicts  between  visual  and  tactile  information. 

6.3.9.  AUDITORY  DISPLAYS 

Auditory  displays  are  also  a “head  up”  source  of 
information  for  pilots.  Although  auditory  signals 
have  been  used  in  crew  station  design  for  some  time, 
to  date  they  have  been  limited  to  single  frequencies 
or  voice  communications,  primarily  presented 
monaurally.  In  one  application,  navigation 
deviations  were  indicated  with  a Morse  code  type 
auditory  signal:  a Morse  code  “A”  for  one  direction 
and  an  “N”  for  the  other.  The  two  frequencies  fused 
into  a steady  tone  of  1,020  Hz  when  the  aircraft  was 
on  course.  Changes  in  both  frequency  and  pulse  rate 
of  an  auditory  signal  have  been  used  to  indicate  key 
points  in  AOA  (30,  40,  and  70  degrees).  Another 
approach  “aurally”  presents  several  flight  parameters 
with  an  acoustic  orientation  instrument.  The 
instrument  displays  airspeed  as  a sound  frequency 
(repetition  rate),  vertical  velocity  by  amplitude 
modulation  rate  (increase  shown  by  increased  pitch), 
and  bank  angle  by  right/left  lateralization  (louder 
signal  in  side  that  is  same  as  direction  of  bank).  This 
display  was  presented  to  pilots  over  earphones,  after 
processing  the  auditory  signal  to  map  with  the  actual 
aircraft  flight  data.  The  results  showed  that  acoustic 
signals  can  be  useful  indicators  of  the  orientation  of 
an  aircraft,  and  interaural  intensity  differences, 
representing  bank  angle,  are  particularly  effective  in 
this  regard  [42]. 

Additional  improvement  in  the  acoustic  orientation 
instrument  might  be  realized  by  using  three- 
dimensional  localized  signals,  rather  than  lateralized 
signals.  This  is  now  possible  due  to  recent 
advancements  that  have  enabled  the  faithful 
reproduction  of  omnidirectional,  complex  auditory 
signals.  This  includes  duplicating  the  interaural 
intensity  difference,  interaural  time  difference  and 
the  direction  dependent  spectral  information  that 
occurs  when  incoming  sounds  impinge  the  head  and 
outer  ear  (pinnae).  The  latter  are  especially 
important  to  externalize  the  sound  to  appear  “outside 
of  the  head.”  To  reproduce  the  dynamic  cue  changes 
that  occur  with  the  pilot’s  head  movement,  some 
type  of  head  tracker  needs  to  be  integrated  with  the 
audio  display.  Head  tracking  enables  the  headphone 
presented  stimuli  to  be  corrected  in  real-time  so  that 
they  are  perceived  by  the  pilot  to  be  at  fixed 
positions  in  physical  space. 
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For  agile  aircraft  operation,  the  combination  of  three- 
dimensional  auditory  displays  with  a HMD/T  may  be 
especially  useful.  The  auditory  cues  could  improve 
situational  awareness  by  informing  the  pilot  that 
critical  visual  information  lies  outside  of  the  current 
visual  field.  The  spatial  auditory  cues  may  even 
indicate  exactly  where  the  information  is  located 
relative  to  the  current  position  of  the  pilot’s  head  (see 
Figure  6.7).  In  a study  which  compared  different 
methods  of  directing  attention  to  peripheral  targets, 
target  acquisition  time  with  three-dimensional  tones 
was  less  than  other  auditory'  signals  (coded  aural 
tone,  speech  cue,  and  three-dimensional  speech  cue) 

[43] ,  In  another  study,  use  of  spatial  information 
from  the  auditory  channel  reduced  search  latencies 
on  the  order  of  100-200  milliseconds.  This 
advantage  increased  as  the  eccentricity  of  the  target 
increased  beyond  the  limits  of  the  central  visual  field 

[44] ,  The  results  of  an  evaluation  on  the  effects  of 
using  localized  auditory  information  to  perform  a 
target  detection  task  using  a HMD  in  a simulation 
study  were  similar.  Subjects  were  able  to  detect 
targets  with  less  overall  head  motion  and  reduced 
head  velocity  [45],  Under  high  acceleration 
environments,  this  may  help  reduce  the  risk  of  neck 
and  shoulder  fatigue  and  injury'.  In  actual  Harrier 
flight  tests,  a three-dimensional  audio  system  was 
particularly  effective  for  azimuth  cueings.  Aviators 
were  able  to  discern  targets  separated  by  12-20 
degrees  [46],  Three-dimensional  elevation  cues, 
however,  did  not  provide  similar  precision,  but  were 
adequate  in  discriminating  two  spatial  levels  (low 
versus  high). 

Speech  intelligibility  and  discrimination  can  also  be 
improved  by  localizing  speech  inputs.  Small  angular 
separation  of  messages  (45  degrees)  has  been  found 
to  greatly  improve  speech  intelligibility.  At  90 
degrees  of  separation,  the  speech  intelligibility  levels 
were  maximized  [46],  A three-dimensional 
communication  separation  system  also  worked  well 
in  Harrier  flight  tests,  aiding  the  copy  of  dual 
message  traffic  [46], 


Figure  6.7.  Schematic  Illustrating  Application  of 
Three-dimensional  Auditory  Display  for  Three- 
dimensional  Awareness  of  Threats  and 
Communications. 


Information  from  spatialized  auditory  cues  can  also 
help  code  system  status  information.  For  example, 
to  aid  tire  pilot  in  understanding  a critical  situation 
and  add  redundancy  to  the  message,  a left  engine 
auditory  fire  alert  message  could  be  displayed  such 
that  it  appears  to  emanate  from  the  left.  The  auditory 
space  can  also  be  used  to  indicate  the  level  of 
urgency  of  an  auditory  warning.  The  most  urgent 
warnings  would  be  presented  so  they  arc  perceived 
inside  the  head,  while  less  urgent  warning  arc 
perceived  to  the  sides  [47], 

In  sum,  three-dimensional  auditory  signals  have  the 
potential  of  being  detected  more  quickly  than  visual 
signals,  and,  at  the  very  least,  relieving  the  pilot’s 
already  over  burdened  visual  workload.  Some 
candidate  agile  aircraft  applications  of  three- 
dimensional  auditory'  displays  include:  1)  alert  pilots 
of  ground  or  aerial  threat  location  and  facilitate 
target  acquisition,  2)  enhance  situational  awareness 
during  air-to-air  combat  by  localizing  voice 
communications.  3)  segregate  multiple  channels  of 
communication  so  as  to  improve  intelligibility, 
discrimination,  and  selective  attention  among  audio 
sources,  and  4)  provide  an  additional  cue  for  location 
or  urgency  of  an  aircraft  system  malfunction.  Before 
these  candidate  applications  can  be  implemented, 
further  research  is  required  to  determine  how  best  to 
exploit  the  capability  to  present  spatial  auditory 
signals  and  the  best  format  for  the  information  to  be 
presented. 

Given  the  flight  envelope  for  agile  aircraft,  auditory 
localization  accuracy  under  varying  levels  of 
sustained  +Gz  acceleration  is  of  interest.  The  results 
from  one  centrifuge  evaluation  [48]  showed  that 
localization  error  did  not  significantly  increase 
between  1 and  5.6  +Gz.  Error  did  significantly 
increase  at  the  7.0  +Gz  level,  although,  this 
performance  decline  can  also  be  related  to  the 
difficulty  making  the  manual  response  required  in 
the  experimental  task.  Localization  performance  in 
agile  aircraft  will  more  likely  be  influenced  by 
factors  already  known  to  have  an  effect  in  ground- 
based  simulations.  First,  auditory  cues  will  need  to 
be  presented  over  headphones,  as  opposed  to  frcc- 
field  localization,  the  latter  providing  more  accurate 
localization.  This  is  not  foreseen  as  an 
insurmountable  problem,  since  localization  errors 
using  headphone  presentation  has  been  reported  as 
low  as  4.4  to  5.9  degrees,  depending  on  the  type  of 
stimulus  [46],  Also,  given  the  large  ficld-of-view,  a 
general  indication  of  a target’s  position  will  greatly 
benefit  the  pilot.  Second,  errors  in  elevation  arc 
larger  than  for  azimuth  [46],  Once  again,  the  pilot 
will  benefit  from  any  veridical  directional 
information,  whether  it  be  solely  azimuth  or  also 
include  a coarse  indication  of  elevation  (c.g..  high 
versus  low).  Third,  and  most  important,  it  is  likely 
that  auditory'  localization  will  be  accomplished  with 
minimal  movement  of  the  head,  since  the  pilot  will 
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most  likely  be  attending  to  the  forward  field-of-view. 
This  impacts  performance  because  movement  of  the 
head  helps  disambiguate  front/back  reversals  by 
tracking  changes  in  the  magnitude  of  the  interaural 
cues  over  time  influenced  by  the  apparent  source 
position.  The  most  common  type  of  reversals  is 
when  sounds  simulated  in  the  front  hemisphere  are 
heard  at  the  mirror  image  position  in  the  rear.  The 
percentage  of  front/back  reversals  can  be  as  high  as 
50%  of  the  classifications.  Until  this  contusion  is 
controlled,  application  of  three-dimensional  auditory 
displays  might  best  be  limited  to  serving  as  a 
redundant  cue.  Also,  further  research  is  required  to 
evaluate  dynamic  auditory  resolution. 

6.4.  HEAD  DOWN  CONTROLS/DISPLAYS 

In  order  for  the  agile  aircraft  pilot  to  keep  the  head 
up  and  out  of  the  cockpit  as  much  as  possible,  head 
down  information  needs  to  be  easily  acquired  and 
head  down  control  operations  need  to  be  quick  to 
complete.  This  presents  a difficult  challenge  to 
designers  - maximizing  the  information  conveyed  or 
inputted  by  head  down  devices  while  minimizing  the 
time  required  for  head  down  viewing.  The  following 
addresses  these  implications  and  presents  some 
candidate  head  down  controls  and  displays. 

6.4.1.  HEAD  DOWN  CONTROL  ISSUES 
Head  down  controls  need  to  be  easily  located, 
grasped,  and  manipulated.  All  the  information  and 
control  devices  needed  for  a particular  set  of 
activities  should  be  in  close  proximity  and  available 
with  less  than  two  key  presses.  Proper  and 
consistent  formats,  abbreviations,  symbol  meaning, 
control  assignments,  procedures  and  rapid  (<  0.2 
seconds)  feedback  need  to  be  employed  so  the  action 
required  and  status  of  control  operation  is  intuitive  to 
the  pilot  [9],  In  addition  to  dedicated  control 
devices,  many  control  functions  are  activated  by 
selecting  a switch  associated  with  a function 
presented  on  a display.  The  functions  associated 
with  each  switch  change  depending  on  tire  flight 
segment  or  task  to  be  performed.  Human-engineered 
design  of  the  required  interactive  sequences  is  key  to 
the  utility  of  these  multifunction  controls  [49], 
Function  selection  using  touch  activated  displays 
(press  display  surface  over  appropriate  label)  has 
proven  to  be  useful  in  ground-based  applications,  but 
operators  must  be  more  attentive  to  visual  and  audio 
feedback  due  to  the  lack  of  kinesthetic  feedback  [50]. 
Selection  of  small  targets  or  closely  spaced  functions 
is  also  difficult,  especially  with  flight  gloves  (one 
pilot  described  as  “Fist  on  Glass”). 

6.4.2.  HEAD  DOWN  DISPLAY  ISSUES 

The  primary  function  of  head  down  displays  is  to 
increase  the  pilot’s  situational  awareness  and  provide 
additional  systems  information.  If  this  information 
continues  to  be  presented  on  numerous  dials, 
indicators,  and  multiple  small  displays,  it  will  be 
very  difficult  for  pilots  to  rapidly  fuse  the 


information  together  to  access  the  situation.  One 
solution  is  to  present  this  information  on  a single 
large  (e.g.,  flat  panel)  display  [3].  Merely  moving  all 
the  information  onto  one  surface  will  not  facilitate 
pilot  performance.  Rather  careful  format  design  is 
required  to  identify  an  integrated  format  that  makes 
it  easy  for  the  pilots  to  determine  what  actions  are 
possible  at  any  moment  and  evaluate  the  current  state 
of  aircraft  systems.  In  other  words,  the  right 
information  in  a useful  format  needs  to  be  presented 
at  the  right  time.  Moreover  the  information  needs  to 
be  presented  in  the  right  location  - any  critical 
information  should  appear  at  the  same  location  all 
the  time. 

With  programmable  displays,  there  is  virtually  no 
limit  as  to  how  information  can  be  presented.  This  is 
a mixed  blessing  because  there  is  a natural  tendency 
to  provide  tire  pilot  with  several  options,  not 
knowing  the  optimal  approach  in  advance.  This  is 
counter-productive,  adding  to  the  pilot’s  visual 
workload  and  cognitive  demands  to  filter  out  the 
required  information.  Display  formats  for  head 
down  displays  (as  well  as  head  up  displays) 
represents  a research  topic  requiring  significant 
attention.  Specific  design  guidelines  and  useful 
metrics  for  managing  the  presentation  of  information 
in  multipage  displays  are  available  in  [51], 

For  map  displays,  the  scale  and  frame  of  reference 
(track  up  or  north  up)  should  be  pilot  selectable.  As 
a default,  a track  up  view  in  which  the  map  display 
rotates  to  match  the  momentary  heading  of  the 
aircraft  is  better  in  that  it  eliminates  the  need  for  the 
pilot  to  do  mental  rotation  [23].  A three-dimensional 
presentation  may  help  provide  information  about  the 
relative  distances  of  objects  from  the  ownship.  To 
date,  though,  there  has  not  been  a consistent 
advantage  with  a three-dimensional  approach  [23], 
Color-coding  lias  been  found  useful  for 

distinguishing  boundaries  and  differentiating 

symbology  sets.  Rather  than  present  sensor  data  on 
different  formats,  an  attempt  should  be  made  to  fuse 
the  information  into  a single  format  or  integrate  and 
code  the  information  so  that  the  pilot  knows  the 
source  of  each  datum. 

Information  presented  on  head  down  displays  is  the 
primarily  source  of  weapon  and  aircraft  systems 
status.  The  information  needs  to  be  limited  to  what 
is  meaningful  or  more  easily  used  by  the  pilot. 
Examples  include  fiiel  in  available  range  format  and 
threats  as  potential  killers  or  not  [1].  For  some 
systems,  a pictorial  presentation  of  tire  information 
may  be  more  intuitive  and  more  quickly  assimilated. 
Figure  6.8  shows  examples  that  were  generated  in  a 
study  to  examine  use  of  pictorial  formats  for  military 
cockpits  [22].  Pictorial  symbols  can  be  used  to 
describe  the  status  of  subsystems  as  well.  For 
example,  a pictorial  representation  of  the  four 
mechanical  fuzing  options  (nose,  tail,  both,  or  none) 
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Figure  6.8.  Example  Pictorial  Display  Formats 
Showing  Systems  Status  [22],  First  two  show 
weapons  and  fuel  status.  Last  two  are  engine 
formats. 

provides  a more  realistic  impression  of  what  is 
actually  happing  with  the  fuzing,  compared  to  an 
alphanumerical  readout.  Color-coding  can  also  be 
very  useful  to  note  operating  ranges  of  a system 
parameter.  Evaluations  are  required  to  determine 
whether  additional  costs  to  provide  pictorial  and 
color  formats  are  merited  in  terms  of  aircrews’ 


performance  and  preference.  The  results  may  differ 
depending  on  the  particular  display  format  [52]. 

The  flight  control  stick  can  also  convey  information 
to  the  pilot.  In  the  past,  stall  warning  systems 
employed  stick  shakers  or  stick  pushers  to  warn 
pilots  of  impending  stall  conditions.  The  pilot’s 
attention  can  be  easily  acquired  by  altering  the 
control  stick’s  force  gradient.  More  recently,  the 
utility  of  the  pilot’s  sense  of  touch  was  examined  in  a 
landing  experiment  that  fed  information  concerning 
lateral  deviations  from  a runway  centerline  into  a 
force  reflecting  control  stick.  The  results  indicated  a 
consistent  advantage  in  performance  and  perceived 
workload  for  the  force  feedback  system,  particularly 
for  landings  conducted  under  heavy  turbulence  [53]. 

6.5.  MULTI-MODAL  CONTROLS/DISPLAYS 

A combination  of  modalities  can  facilitate  the 
information  exchange  between  the  agile  aircraft 
cockpit  and  pilot.  However,  there  is  also  a danger  in 
imposing  an  additional  load  on  the  pilot  to  remember 
the  steps  used  to  employ  different  controllers  or 
burdening  the  pilot  with  superfluous  stimulation  with 
multi-modal  displays.  It  is  only  through  research 
that  the  optimal  control  and  display  configurations 
can  be  identified  for  specific  tasks/applications. 

6.5.1.  MULTI-MODAL  CONTROL  ISSUES 
Just  as  operators  with  desktop  computers  can 
navigate  with  a variety  of  controls,  it  is  possible  to 
implement  aircraft  system  such  that  several  control 
modalities  can  be  used  for  a single  control  action. 
This  mapping  approach  provides  the  pilot  with 
increased  flexibility:  a)  the  pilot  may  have  individual 
preferences  for  specific  controls,  b)  a temporary  task 
or  environmental  condition  may  deem  one  controller 
more  efficient  than  another  (e.g.,  eye-based  control 
when  manual  selection  is  difficult  under  high 
acceleration  or  positive  pressure  breathing  interferes 
with  speech-based  commands),  and  d)  should  one 
control  device  malfunction,  the  pilot  can  use  a 
different  control. 

A multi-modal  approach  is  also  useful  when  two  or 
more  controls  arc  integrated  such  that  they  are  used 
together  to  perform  a task.  In  one  integration 
approach,  a control  technology  that  cannot  perform  a 
particular  function  alone  can  be  used  to  improve  the 
performance  of  another  control.  For  instance,  eye 
line-of-sight  data  might  be  used  to  enhance  speech 
processing  by  restricting  the  vocabulary  search  to  the 
most  probable  verbal  commands  associated  with  the 
current  gaze  point.  In  another  type  of  integration, 
two  or  more  control  devices  operate  in  parallel  to 
increase  the  accuracy  or  reliability  of  a control  action 
(lip  movement  data  when  used  together  with  acoustic 
signals  can  improve  speech  recognition  compared  to 
either  one  alone).  In  a third  integration  type,  controls 
are  mapped  to  different  subcomponents  of  a task. 
For  example,  the  pilot  can  use  eye  gaze  to  designate 
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a waypoint  on  a map  display  and  a voice  command 
for  a consent  response,  commanding  the  navigation 
system  to  update  the  mission  plan.  The  use  of  both 
control  devices  capitalizes  on  the  ability  of  eye  gaze 
to  rapidly  designate  a position  on  two-dimensional 
surfaces  and  voice  commands  to  quickly  initiate  an 
action.  In  fact,  eye  and  voice  systems  can  replace  or 
augment  conventional  controls  for  many  interactions 
with  aviation  displays  (e.g.  configure  displays,  tailor 
displayed  information,  retrieve  information,  and 
input  information)  [54], 

6.5.2.  MULTI-MODAL  DISPLAY  ISSUES 
Multi-modal  displays  may  be  more  effective  in 
warning  the  pilot  of  an  aircraft  system  malfunction 
or  an  impending  threat.  In  one  experiment,  visual 
icons  and  verbal  warning  messages  were  used  singly 
and  in  combination  and  the  results  showed  a 
significant  decrease  in  response  latencies  when 
correlated  bimodal  information  was  provided,  as 
compared  to  either  unimodal  alert  [55],  In  another 
experiment,  both  a three-dimensional  tactical  (visual) 
radar  display  and  a three-dimensional  auditory 
display  were  presented  to  provide  the  pilot  with 
information  about  the  target  aircraft.  The  radar 
display  showed  the  target’s  relative  speed  and 
whether  it  was  above  or  below  ownsliip.  The 
auditory  display  showed  the  direction  of  the  target  to 
ownship.  The  displays  also  differed  with  respect  to 
frame  of  reference.  The  radar  display  was  outside- 
in,  indicating  the  relative  position  of  tire  target  as 
seen  from  above  and  the  auditory  display  was  inside- 
out,  indicating  position  relative  to  the  subject’s  head. 
The  results  showed  that  both  displays,  when  used 
individually,  reduced  search  time  [56],  However, 
when  the  two  modalities,  visual  and  auditory,  were 
used  simultaneously,  search  time  was  reduced  more. 

Multi-modal  displays  can  also  help  overcome  the 
inherent  limitations  of  display  technologies  when 
used  individually.  For  instance,  target  detection 
performance  has  been  found  to  be  poorer  with  a 
HMD  compared  to  a full  field-of-view  visual 
condition  [57],  The  results  of  follow-on  research 
suggest  that  a three-dimensional  auditory  display  can 
be  effective  in  mitigating  the  negative  effects 
associated  with  performing  a visual  target  detection 
task  with  a HMD  [45],  Another  example  where  two 
modalities  could  complement  each  other  is  three- 
dimensional  auditory  displays  and  tactile  displays. 

6.6.  INTELLIGENT  INTERFACES 

6.6.1.  ADAPTIVE  INTERFACES 
The  use  of  computer-driven  controls  and  displays  in 
agile  aircraft  cockpits  offers  the  opportunity  to 
include  intelligent  interfaces  which  help  the  pilot 
acquire  information  and  execute  decisions.  This 
would  provide  more  time  for  the  pilot  to  control  the 
aircraft  and  think  about  decisions  that  must  be  made. 


To  meet  this  objective,  the  displays  must  be 
configured  to  provide  information  salient  to  the 
specific  situation  being  addressed  by  the  pilot  and 
the  controls  must  facilitate  the  pilot’s  response.  The 
use  of  tailoring  has  already  been  introduced  - only 
information  previously  determined  appropriate  for 
the  current  flight  phase  is  presented.  This  tailoring  is 
a result  of  an  explicit  control  input  by  the  pilot  (e.g., 
selection  of  a flight  mode  switch).  However,  it  is 
likely  that  the  pilot  would  benefit  from  variations  in 
the  control/display  configuration  for  specific  tasks 
within  a single  flight  segment.  Rather  than  have  the 
pilot  continually  commanding  the  system  to  make 
such  changes  (and  in  cases  where  the  pilot  is  over 
loaded  or  incapacitated),  it  is  desirable  to  have 
dynamically  adaptive  interfaces  that  change  the 
display  and/or  control  characteristics  in  real  time 
[58]  (see  Figure  6.9).  These  changes  are  initiated  by 
predetermined  triggers: 

external  (changes  in  mission,  tactical 
constraints,  threats,  and  aircraft  systems 
(hydraulic  failure)), 

internal  (physiological)  indices  (measurable 
aspects  of  the  neurophysiology  that  index 
changes  in  the  pilot’s  physical  and  cognitive 
states),  and 

behavioral  indices  (overt  behaviors  executed  by 
the  pilot  (eye  gaze  point,  control  activity,  etc.). 

Besides  choosing  and  validating  the  triggers  and 
decision  rules  that  initiate  the  adaptations,  the 
specific  modifications  to  be  made  to  displays  and 
controls  in  each  instance  must  be  identified. 
Implementation  of  adaptive  interfaces  is  also  a 
challenge  due  to  the  real-time  timing  constraints  and 
the  need  to  analyze  continuous  parallel  input  streams 
from  numerous  sources.  To  ensure  that  the  candidate 
adaptive  interfaces  are  indeed  a benefit  to  the  pilot, 
evaluations  are  required.  The  goal  is  to  provide  the 
agile  aircraft  pilot  with  the  right  information,  at  the 
right  time,  and  in  the  right  location  for  optimal 
performance  and  mission  success.  On  the  other 
hand,  there  are  potential  problems  with  impeding  the 
pilot’s  cognitive  momentum  and  causing  confusion 
by  changing  information.  The  lack  of  consistency 
could  also  interfere  with  the  pilot’s  skilled-based 
behavior.  However,  the  results  of  one  simulation 
suggest  that  dynamic  changes  in  displays  or  controls 
will  not  interfere  with  the  development  or  execution 
of  skilled  behavior  [59].  This  experiment  utilized 
three  interface  conditions:  conventional,  advanced 
(flight  director  display  and  force  reflecting  stick), 
and  adaptive  (which  switched  between  the 
conventional  and  advanced,  depending  on  pilot 
performance  on  a navigation  task).  The  need  for 
further  evaluation  was  indicated,  though,  that  utilizes 
a more  demanding  task  environment  and  more 
complex  mechanisms  to  trigger  adaptive  changes. 
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Figure  6.9.  Illustration  of  Adaptive  Interface  Components  in  Crew  Station  Design. 


Adaptive  interfaces  can  also  capitalize  on  the 
human’s  capability  for  parallel  processing  across 
sensory  modalities.  If  a workload  assessor 
determines  tliat  the  pilot’s  visual  clianncl  is 
saturated,  then  a high-urgency  display  element  that 
would  nominally  be  presented  on  a visual  display 
(e.g.,  approach  of  a G-limit)  could  be  presented  via 
the  auditory  system  or  via  force  feedback  in  the 
control  stick.  If  it  is  determined  tliat  tire  pilot  is 
heavily  engaged  in  some  activity  (evading  an  enemy) 
and  does  not  have  tire  resources  to  attend  to  another 
task  (activate  electronic  counter  measures), 
intelligent  systems  could  automatically  perform  the 
task  and  inform  tire  pilot  of  its  completion.  In  this 
case,  tire  intelligent  system  is  not  only  changing  the 
interfaces  but  also  accomplishing  a task  for  the  pilot. 
This  role  of  automation  in  crew  system  design  raises 
additional  issues,  which  are  discussed  below. 

6.6.2.  AUTOMATION 

Automation  of  some  crew  station  tasks  can  certainly 
help  reduce  the  workload  facing  the  agile  aircraft 
pilot.  However,  problems  with  automation  can  arise 
due  to  “clumsy”  use  of  tliis  technology.  If  the  entire 
pilot/system  operation  is  not  considered,  there  is  a 
likelihood  of  automation  being  activated  at  a time 
when  it  is  least  needed  and  hindering  performance  in 


situations  where  it  is  greatly  needed.  Even  more 
common  is  the  failure  to  provide  the  pilot  with 
adequate  feedback  on  the  status  and  behavior  of  the 
automated  system  or  task,  which  affects  the  pilot’s 
ability  to  maintain  situational  awareness.  The  pilot 
needs  to  be  able  to  maintain  a mental  model  of  both 
the  monitored  process  and  the  status  of  the  intelligent 
system,  especially  when  multiple  dynamic  systems 
are  in  operation  as  is  the  case  in  aviation.  Special 
attention  should  be  given  to  the  level  of  feedback. 
The  complexity'  of  modem  systems  makes  it 
impossible  and  undesirable  to  display  every  data 
item,  but  a minimum  level  of  information  is  desirable 
to  keep  the  pilot  on  line,  so  tliat  decisions  can  be 
made  when  needed.  Also,  the  level  of  information 
provided  to  the  pilot  may  be  context-dependent. 
Lastly,  there  are  instances  where  automating  a task  is 
not  in  the  best  interests  of  the  pilot.  For  example, 
having  the  pilot  a passive  occupant  during  automatic 
guns  aiming  or  automated  missile  avoidance  can 
increase  the  pilot’s  disorientation  and  sickness 
during  abrupt  maneuver  changes. 

Some  of  these  problems  arise  because  the 
environment  and  workload  characteristics  of  the 
cockpit  arc  so  complex  and  dynamic.  What  may  be 
an  optimal  automation  scheme  for  one  flight 
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segment/task  may  be  totally  inappropriate  for 
another.  Therefore,  automation  also  needs  to  be 
dynamic  or  adaptive,  with  the  goal  of  maintaining  an 
optimal  division  of  labor  between  the  pilot  and  the 
aircraft  [60],  In  other  words,  the  automation  needs  to 
be  flexible  and  responsive  to  pilot  and  task  demands 
and  the  same  triggers  used  in  adaptive  interfaces  are 
also  useful  for  adaptive  automation.  With  this  pilot 
centered  approach,  there  should  be  fewer  difficulties 
with  automation  induced  difficulties  in  monitoring 
and  maintaining  situational  awareness  because  the 
pilot  is  kept  more  involved.  Tasks  requiring 

judgement,  multi-sensoiy  information  gathering, 
hypothetical  reasoning,  and  contingency  reaction  are 
best  suited  with  the  pilot  in  the  loop.  There  are, 
however,  some  ongoing  “housekeeping”  tasks  that 
can  be  automated;  tasks  that  require  accurate 
responses,  fastidious  and  repetitive  actions,  and 
exhaustive  calculations  are  good  candidates  for 
automation.  The  following  are  example  candidate 
applications  of  automation: 

if  an  engine  failure  is  detected,  perform 
correction  and  concurrently  notify  the  pilot 
manage  fuel  and  hydraulic  systems,  but  give 
high  level  information  to  pilot  (range, 
malfunctions,  etc.) 

perform  appropriate  actions  for  battle  damages 
and  inform  pilot  if  operational  capabilities  or 
flight  performance  affected 
manage  navigation  systems,  but  store  data  for 
call  up  by  the  pilot 

assess  situation,  manage  sensors  and  attach 
confidence  indicators  to  fused  and  correlated 
outputs 

analyze  target  to  provide  identification, 
performance  capabilities  and  optimum 
engagement  parameters 

deploy  aircraft  defensive  measures  when  pilot  is 
busy  accomplishing  a popup  weapon  delivery 
sequence  [61], 

Besides  adapting  automation  to  the  pilot  and  task 
demands,  a human  centered  approach  calls  for 
obtaining  the  consent  of  the  pilot  (or  requiring  a 
command  from  the  pilot)  before  initiating  an 
automated  action.  This  pilot  preferred  approach  is 
referred  to  as  “management  by  consent”  - 
automation  cannot  take  action  unless  and  until 
explicit  pilot  consent  has  been  received  [62],  Since 
there  are  numerous  instances  where  automation 
could  play  a role,  requiring  a consent  response  for 
every  automated  action  is  also  unreasonable.  It  is 
recommended  that  the  pilot  input  a nominal  set  of 
rules  to  be  used  by  the  automation  system  for  the 
majority  of  tasks.  For  each  task,  die  pilot  should 
indicate  preferences  on  whether  the  automadon 
system  should:  always  perform  die  task,  sometimes 
perform  the  task,  perform  the  task  and  notify  the 
pilot,  or  ask  for  permission  to  perform  die  task. 
Having  the  pilot  tailor  die  automation  system  before 


die  mission  helps  minimize  the  chance  of  interfering 
with  workload  on  other  simultaneous  tasks. 

The  term  “management  by  exception”  refers  to 
instances  where  die  system  takes  over  sometimes. 
For  instance,  automadon  systems  can  perform  less 
critical  tasks  on  their  own  when  it  is  detected  that  the 
pilot  is  suffering  from  demanding  time  pressures  and 
workload.  Of  course,  the  pilot  maintains  an  option  to 
override  this  automation.  There  are  also  instance 
where  pilot  consent  may  not  be  practical  (e.g.,  pilot 
injured)  and  function  changes  may  need  to  be 
implemented  directly  by  the  adaptive  system. 

Given  die  number  of  aircraft  systems  and 
corresponding  procedures  and  tasks  involved, 
research  is  needed  to  decide  how  tasks  should  be 
shared  between  die  pilot  and  aircraft,  how  much 
autonomy  and  authority  should  be  given  to  each,  and 
how  agreements  and  commitments  to  actions  can  be 
negotiated  between  the  two.  Certainly,  the  degree  to 
which  automation  is  successful  in  agile  aircraft  is  a 
function  of  the  degree  to  which  there  is  coordination 
between  die  pilot  and  die  automation  system  [61], 

6.7.  SUMMARY 

The  issues  raised  in  this  lecture  pertaining  to 
controls,  displays,  and  intelligent  interfaces  illustrate 
opportunities  for  enhancing  the  cooperative 
interaction  between  die  pilot  and  the  aircraft,  with 
the  ultimate  goal  of  achieving  pilot-cockpit 
symbiosis.  Moreover,  the  importance  of  the  pilot- 
cockpit  interface  to  die  successful  exploitation  of 
agile  airframes,  agile  weapons,  and  agile  systems  lias 
been  demonstrated.  It  is  clear  tiiat  considering 
ergonomics  in  crew  station  design  is  key  to  the 
success  of  agile  aircraft. 

6.7.1.  THE  GOOD  NEWS 

The  results  of  the  pilot  interviews  and  die  reviews  of 
diis  Working  Group  show  that  drastic  changes  in  the 
crew  station  design  hardware  are  not  needed  for 
agile  aircraft.  For  die  most  part,  near  term  control 
and  display  suites  (Figure  6.10),  current  systems 
along  widi  die  advances  diat  are  nearing  transition 
(e.g,  HMD/T),  are  adequate.  Even  diough 
modifications  in  formatting  and  configuration  are 
required  to  address  specific  agile  aircraft  issues  (e.g., 
presenting  flight  path  information  when  at  a high 
AOA),  most  are  easy  to  implement  since  so  much  of 
die  hardware  is  computer  driven.  A recent  simulated 
air  combat  study  demonstrated:  1)  the  feasibility  of 
implementing  many  of  these  advanced  concepts,  and 
2)  these  advanced  concepts  can  result  in  statistically 
significant  advantages,  despite  the  fact  that  die 
subjects  (pilots  from  diree  NATO  countries)  were 
more  experienced  with  conventional  crew  stations 
[63].  This  evaluation  assessed  both  a conventional 
cockpit  (F-16/F-15  type  cockpit  displays)  and  a 
virtually  augmented  cockpit  (HMD/T,  pictorial 
formats,  color  coding,  diree-dimensional  audio 
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NEAR  TERM 

Head  Mounted  Tracker/Display  System 
Integrated  Flight  Control  System 
Hands-on-Throttle-and-Stick 
Voice  Control  for  secondary  tasks 
Improved  Display  Formats: 

color,  pictorial,  and  sensor  fusion 

FAR  TERM 

Visual  Peripheral  & 3-D  Displays 

Tactile  Displays 

Auditory  3-D  Displays 

Eye  Control 

Gesture  Control 

Bio-potential  Control 

Multi-modal  Displays/Controls 

Adaptive  Displays/Controls 

Figure  6.10.  Candidate  Near  and  Far  Term  Displays 
and  Controls  for  Agile  Aircraft. 

cueing  for  the  radar  warning  receiver,  and  a ground 
collision  avoidance  system)  using  objective  and 
subjective  measures.  The  findings  indicated  that  the 
new  design  not  only  resulted  in  superior  mission 
performance,  but  also  did  so  with  less  workload  and 
enhanced  situational  awareness.  In  general,  those 
aspects  of  the  mission  that  relied  on  target 
identification  and  maintenance  of  tactical  position 
relative  to  the  target  appeared  to  be  the  most 
positively  affected  by  the  advanced  crew  station 
design. 

Therefore,  there  is  good  news  that  significant 
investments  in  control  and  display  hardware 
development  are  not  required  to  meet  the  pilot- 
vehicle  interface  requirements  of  agile  aircraft.  Far 
term  developments  (Figure  6.10)  that  provide  the 
pilot  with  new  capabilities  are,  though,  certainly 
welcomed  candidates  for  agile  aircraft  application. 

6.7.2.  THE  BAD  NEWS 
The  results  of  this  effort  showed  that  the  mental 
workload  involved  with  information  management  in 
crew  station  operation  is  a limiting  factor  for  agile 
aircraft  operation.  In  order  to  achieve  full 
operational  performance  in  agile  aircraft,  the  pilot 
must  be  able  to  perform  several  simultaneous 
functions:  fly  the  aircraft,  maintain  situational 
awareness  of  the  total  air  battle  scenario, 
communicate  with  friendly  forces,  plan  attacks,  fly 
complex  attack  maneuvers,  control  aiming  and 
release  of  multiple  weapons,  manage  onboard 
systems,  organize  self  defense  against  arriving 
threats,  and  perform  high  acceleration  escape 
maneuver  for  threat  avoidance.  Given  the  increasing 
number  of  systems  involved  in  completing  these 
tasks  and  the  myriad  of  control  options  available,  it 
is  not  difficult  to  understand  how  the  pilot  can  be 
overwhelmed.  Rather  than  helping  the  pilot  with 
these  added  capabilities,  the  design  may  in  fact  be 


hindering  the  pilot’s  ability  to  execute  and  survive 
the  mission. 

The  bad  news  is  that  the  current  approaches  for 
pilot-vehicle  interfaces  do  not  support  fast 
assimilation  of  information  and  control  actuation. 
Simply,  the  right  information  is  not  being  provided  at 
the  right  time  in  the  right  location.  It  is  this 
inadequate  information  flow  between  the  pilot  and 
the  aircraft  that  is  the  limiting  factor  in  the 
performance  of  agile  aircraft.  Therefore,  a 
significant  investment  is  needed  to  conduct  the 
human  factors  engineering,  task  analysis,  design 
iteration,  and  evaluation  needed  to  identify  how  the 
pilot-vehicle  interface  needs  to  be  improved  to 
support  pilot/airframc/wcapons/systcms  information 
exchange  [9],  Compared  to  past  aviation  ergonomic 
studies,  this  needed  research  will  be  much  more 
difficult  to  conduct.  As  the  complexity  and 
dynamics  of  the  systems  increase,  so  do  the 
ergonomic  challenges  to  consider  new  styles  of 
interaction.  New  requirements  arc  levied  on  user 
interface  software  and  user  communication  dialogue 
in  order  to  handle  and  describe  complex  and 
substantial  input/output  processing,  simultaneous 
parallel  inputs,  continuous  inputs  and  outputs, 
imprecise  inputs,  and  timing  constraints.  Thus, 
designers  are  faced  with  both  a great  challenge  and 
opportunity  to  realize  crew  station  designs  that  will 
truly  enhance  the  operation  of  agile  aircraft. 
Hopefully,  the  concepts  and  technologies  described 
in  this  lecture  will  assist  in  this  creation  of  a pilot- 
cockpit  symbiosis  for  agile  aircraft  applications. 
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